SherbyNet - /page/wikibook/entry.mu

Micron Document

<!DOCTYPE html>

<title>Concentration parameter</title>

</head>

<span id="openzim-page-title" class="mw-page-title-main"><span class="mw-page-title-main">Concentration parameter</span></span>

</h1>

</header>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hatnote{font-style:italic}.mw-parser-output div.hatnote{padding-left:1.6em;margin-bottom:0.5em}.mw-parser-output .hatnote i{font-style:normal}.mw-parser-output .hatnote+link+.hatnote{margin-top:-0.5em}@media print{body.ns-0 .mw-parser-output .hatnote{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div role="note" class="hatnote navigation-not-searchable">For the inverse of a covariance matrix, see <a href="Concentration_matrix" class="mw-redirect" title="Concentration matrix">Concentration matrix</a>.</div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .ambox{border:1px solid #a2a9b1;border-left:10px solid #36c;background-color:#fbfbfb;box-sizing:border-box}.mw-parser-output .ambox+link+.ambox,.mw-parser-output .ambox+link+style+.ambox,.mw-parser-output .ambox+link+link+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+style+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+link+.ambox{margin-top:-1px}html body.mediawiki .mw-parser-output .ambox.mbox-small-left{margin:4px 1em 4px 0;overflow:hidden;width:238px;border-collapse:collapse;font-size:88%;line-height:1.25em}.mw-parser-output .ambox-speedy{border-left:10px solid #b32424;background-color:#fee7e6}.mw-parser-output .ambox-delete{border-left:10px solid #b32424}.mw-parser-output .ambox-content{border-left:10px solid #f28500}.mw-parser-output .ambox-style{border-left:10px solid #fc3}.mw-parser-output .ambox-move{border-left:10px solid #9932cc}.mw-parser-output .ambox-protection{border-left:10px solid #a2a9b1}.mw-parser-output .ambox .mbox-text{border:none;padding:0.25em 0.5em;width:100%}.mw-parser-output .ambox .mbox-image{border:none;padding:2px 0 2px 0.5em;text-align:center}.mw-parser-output .ambox .mbox-imageright{border:none;padding:2px 0.5em 2px 0;text-align:center}.mw-parser-output .ambox .mbox-empty-cell{border:none;padding:0;width:1px}.mw-parser-output .ambox .mbox-image-div{width:52px}@media(min-width:720px){.mw-parser-output .ambox{margin:0 10%}}@media print{body.ns-0 .mw-parser-output .ambox{display:none!important}}

/* end https://en.wikipedia.org/ */

</style>

<p>In <a href="Probability_theory" title="Probability theory">probability theory</a> and <a href="Statistics" title="Statistics">statistics</a>, a <b>concentration parameter</b> is a special kind of <a href="Numerical_parameter" class="mw-redirect" title="Numerical parameter">numerical parameter</a> of a <a href="Parametric_family" title="Parametric family">parametric family</a> of <a href="Probability_distribution" title="Probability distribution">probability distributions</a>. Concentration parameters occur in two kinds of distribution: In the <a href="Von_Mises%E2%80%93Fisher_distribution" title="Von Mises–Fisher distribution">Von Mises–Fisher distribution</a>, and in conjunction with distributions whose domain is a probability distribution, such as the <a href="Symmetric_Dirichlet_distribution" class="mw-redirect" title="Symmetric Dirichlet distribution">symmetric Dirichlet distribution</a> and the <a href="Dirichlet_process" title="Dirichlet process">Dirichlet process</a>. The rest of this article focuses on the latter usage.

</p><p>The larger the value of the concentration parameter, the more evenly distributed is the resulting distribution (the more it tends towards the <a href="Uniform_distribution_(continuous)" class="mw-redirect" title="Uniform distribution (continuous)">uniform distribution</a>). The smaller the value of the concentration parameter, the more sparsely distributed is the resulting distribution, with most values or ranges of values having a probability near zero (in other words, the more it tends towards a distribution concentrated on a single point, the <a href="Degenerate_distribution" title="Degenerate distribution">degenerate distribution</a> defined by the <a href="Dirac_delta_function" title="Dirac delta function">Dirac delta function</a>).

</p>

<div class="mw-heading mw-heading2"><h2 id="Dirichlet_distribution">Dirichlet distribution</h2></div>

<p>In the case of multivariate Dirichlet distributions, there is some confusion over how to define the concentration parameter. In the topic modelling literature, it is often defined as the sum of the individual Dirichlet parameters,<sup id="cite_ref-1" class="reference"><a href="#cite_note-1"><span class="cite-bracket">[</span>1<span class="cite-bracket">]</span></a></sup> when discussing symmetric Dirichlet distributions (where the parameters are the same for all dimensions) it is often defined to be the value of the single Dirichlet parameter used in all dimensions. This second definition is smaller by a factor of the dimension of the distribution.

</p><p>A concentration parameter of 1 (or <i>k</i>, the dimension of the Dirichlet distribution, by the definition used in the topic modelling literature) results in all sets of probabilities being equally likely, i.e., in this case the Dirichlet distribution of dimension <i>k</i> is equivalent to a uniform distribution over a <a href="Standard_simplex" class="mw-redirect" title="Standard simplex"><i>k-1</i>-dimensional simplex</a>. This is <i>not</i> the same as what happens when the concentration parameter tends towards infinity. In the former case, all resulting distributions are equally likely (the distribution over distributions is uniform). In the latter case, only near-uniform distributions are likely (the distribution over distributions is highly peaked around the uniform distribution). Meanwhile, in the limit as the concentration parameter tends towards zero, only distributions with nearly all mass concentrated on one of their components are likely (the distribution over distributions is highly peaked around the <i>k</i> possible <a href="Dirac_delta_distribution" class="mw-redirect" title="Dirac delta distribution">Dirac delta distributions</a> centered on one of the components, or in terms of the <i>k</i>-dimensional simplex, is highly peaked at corners of the simplex).

</p>

<div class="mw-heading mw-heading2"><h2 id="Sparse_prior">Sparse prior</h2></div>

<p>An example of where a sparse prior (concentration parameter much less than 1) is called for, consider a <a href="Topic_model" title="Topic model">topic model</a>, which is used to learn the topics that are discussed in a set of documents, where each "topic" is described using a <a href="Categorical_distribution" title="Categorical distribution">categorical distribution</a> over a vocabulary of words. A typical vocabulary might have 100,000 words, leading to a 100,000-dimensional categorical distribution. The <a href="Prior_distribution" class="mw-redirect" title="Prior distribution">prior distribution</a> for the parameters of the categorical distribution would likely be a <a href="Symmetric_Dirichlet_distribution" class="mw-redirect" title="Symmetric Dirichlet distribution">symmetric Dirichlet distribution</a>. However, a coherent topic might only have a few hundred words with any significant probability mass. Accordingly, a reasonable setting for the concentration parameter might be 0.01 or 0.001. With a larger vocabulary of around 1,000,000 words, an even smaller value, e.g. 0.0001, might be appropriate.

</p>

<ul><li><a href="Dirichlet_distribution" title="Dirichlet distribution">Dirichlet distribution</a></li>

<li><a href="Dirichlet_process" title="Dirichlet process">Dirichlet process</a></li>

<li><a href="Pitman%E2%80%93Yor_process" title="Pitman–Yor process">Pitman–Yor process</a></li>

<li><a href="Location_parameter" title="Location parameter">Location parameter</a></li>

<li><a href="Scale_parameter" title="Scale parameter">Scale parameter</a></li></ul>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist">

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite id="CITEREFWallachIain_MurrayRuslan_SalakhutdinovDavid_Mimno2009" class="citation conference cs1"><a href="Hanna_Wallach" title="Hanna Wallach">Wallach, Hanna M.</a>; Iain Murray; Ruslan Salakhutdinov; David Mimno (2009). "Evaluation methods for topic models". <i>Proceedings of the 26th Annual International Conference on Machine Learning</i>. ICML '09. New York, NY, USA: ACM. pp. <span class="nowrap">1105–</span>1112. <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.149.771">10.1.1.149.771</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F1553374.1553515">10.1145/1553374.1553515</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-60558-516-1</bdi>.</cite></span>

</li>

</ol></div></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2023-12-28" href="https://en.wikipedia.org/wiki/?title=Concentration_parameter&oldid=1192259860">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>